Skip to content

feat(chart): add server-side column sorting for table and drill-detail views#40907

Open
ponichWebKnight wants to merge 2 commits into
apache:masterfrom
ponichWebKnight:feature/chart-table-data-sort
Open

feat(chart): add server-side column sorting for table and drill-detail views#40907
ponichWebKnight wants to merge 2 commits into
apache:masterfrom
ponichWebKnight:feature/chart-table-data-sort

Conversation

@ponichWebKnight

Copy link
Copy Markdown

SUMMARY

Adds header-click server-side sorting to the paginated/truncated table views so
sorting reflects the full dataset rather than only the rows already loaded on the
client. Previously, clicking a column header in these views either did nothing useful or
only reordered the current page.

What changed

  • "View as table" results pane (DataTablesPaneGridTable): clicking a column
    header now injects an orderby into the chart-data request and re-queries the server.

    • The server round-trip is skipped when the result already fits within the row
      limit (client-side sort is exact) — so small results stay instant.
    • It is also skipped for queries with post-processing (pivot / cum / rolling).
      orderby + row_limit apply to the raw SQL before post-processing, which would
      change the rows feeding those operations and produce incorrect values (e.g. a
      cumulative total computed over a truncated subset). Those views fall back to
      client-side sorting of what the chart actually produced.
    • The results pane stays mounted during a re-sort refetch, so the grid keeps its sort
      state/indicator instead of resetting.
  • Multi-column sort: the chart-data API now validates only that each orderby
    direction is a boolean and allows multiple sort columns; the table plugins'
    (the metric-based default is only applied when no orderby is provided).

  • Drill to detail (DrillDetailPane): column-header sort is wired through to
    /datasource/samples as a server-side orderby.

    • SamplesPayloadSchema accepts an orderby field.
    • _get_drill_detail preserves a caller-supplied orderby, defaulting to the first
      column only when none is given (previously it unconditionally forced the first
      column, ignoring the request).
    • orderby is omitted from the COUNT(*) query (ordering a bare count by a data
      column is meaningless and errors on stricter databases).

Design decisions / rationale

  • Server-side sort is only enabled where the displayed columns map directly to the SQL
    result; post-processed charts are intentionally left to client-side sort to avoid
    returning incorrect data.
  • Sort-column validation is kept minimal at the API layer; unknown sort columns are
    rejected deeper in the query builder (get_sqla_query raises on unresolved columns),
    so injection-style column names cannot reach SQL.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Screenshot 2026-06-09 at 18 14 20 Screenshot 2026-06-09 at 18 09 34 Screenshot 2026-06-09 at 18 09 41 Screenshot 2026-06-09 at 18 09 59

TESTING INSTRUCTIONS

Automated:

  • Backend unit tests:
    • pytest tests/unit_tests/charts/data/test_api.py
    • pytest tests/unit_tests/views/datasource/utils_test.py
    • pytest tests/unit_tests/common/test_query_actions.py
  • Backend integration test:
    • pytest tests/integration_tests/datasource_tests.py -k samples_with_orderby

Manual:

  1. Results pane (raw/non-post-processed chart): open a chart whose result exceeds the
    row limit, open the "View as table" menu, click a column header → confirm the
    request re-fires with queries[0].orderby and the rows re-order by the full dataset
    (top-N changes), cycling asc → desc → off.
  2. Results pane (post-processed chart, e.g. Big Number with a cumulative trend): click
    a column header → confirm sorting happens client-side with no server request and
    the displayed values stay correct (no truncated cumulative totals).
  3. Drill to detail: right-click a chart → Drill to detail, click a column header →
    confirm the /datasource/samples request includes orderby, rows sort server-side,
    and sorting persists while paging.
  4. Multi-column sort: Shift-click a second header in the results table → confirm
    multiple orderby entries are sent and accepted.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

ponichWebKnight and others added 2 commits June 9, 2026 17:05
…l views

Enable header-click sorting that re-queries data from the server for
paginated/truncated table views, so sorting reflects the full dataset
rather than only the rows already on screen.

- View as table (results pane): sort the GridTable header server-side by
  injecting `orderby` into the chart data request. Skip the round trip when
  the result fits within the row limit (client sort is already exact) or when
  the query uses post-processing (pivot/cum/rolling), where SQL-level orderby
  would corrupt the result. Keep the pane mounted on refetch so the grid
  retains its sort state.
- Multi-column sort: the chart data API validates only the sort direction and
  allows multiple orderby columns; the table plugins' buildQuery no longer
  overrides an explicitly supplied orderby in aggregate mode.
- Drill detail: wire column-header sort through to /datasource/samples.
  SamplesPayloadSchema accepts `orderby`, `_get_drill_detail` preserves a
  supplied orderby (defaulting to the first column only when none is given),
  and `orderby` is omitted from the COUNT(*) query.
- Tests: unit coverage for the sort validator, `get_samples` orderby handling
  and `_get_drill_detail`; integration test for sorted samples.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: AI-MSL CloudGeometry Platform <noreply@cloudgeometry.com>
@dosubot dosubot Bot added api Related to the REST API change:backend Requires changing the backend change:frontend Requires changing the frontend dashboard:drill-to-detail explore:sort Related to sorting in Explore viz:charts:table Related to the Table chart labels Jun 9, 2026
@github-actions github-actions Bot added plugins and removed change:backend Requires changing the backend change:frontend Requires changing the frontend viz:charts:table Related to the Table chart explore:sort Related to sorting in Explore dashboard:drill-to-detail labels Jun 9, 2026
// `orderby` + `row_limit` are applied to the raw SQL *before* post-processing,
// which changes the rows that feed those operations and corrupts the result.
// In that case we fall back to client-side sorting of what the chart produced.
const [hasPostProcessing, setHasPostProcessing] = useState(false);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Initializing post-processing detection to false temporarily enables server-side sorting before async detection finishes, which can send incorrect orderby + row_limit requests for post-processed queries. Start in a disabled/unknown state and enable server sort only after detection confirms no post-processing. [incorrect condition logic]

Severity Level: Major ⚠️
- ❌ Post-processed charts can briefly issue unsafe sorted queries.
- ⚠️ Early header clicks may show incorrect cumulative/rolling values.
- ⚠️ Behavior contradicts documented intent to disable server sort.
Steps of Reproduction ✅
1. Configure a chart whose query uses post-processing (e.g., a cumulative or rolling
operation) so that the backend query context includes a non-empty `post_processing` list
alongside `orderby`, as reflected in
`tests/unit_tests/common/test_query_context_processor.py:1086-1092` (showing `"orderby":
[("Net Amount In", False)], "post_processing": [...]` in the same query).

2. Open the Explore results data panel for that chart: `DataTablesPane` instantiates
`useResultsPane` (`DataTablesPane.tsx:185-198`), which initializes `hasPostProcessing` to
`false` (`useResultsPane.tsx:73`) and immediately kicks off two effects: one that fetches
results via `getChartDataRequest` (`useResultsPane.tsx:108-147`) and one that
asynchronously detects `post_processing` by calling `buildV1ChartDataPayload` and updating
`hasPostProcessing` based on `payload.queries[].post_processing`
(`useResultsPane.tsx:155-186`).

3. As soon as the first result set comes back, `isLoading` is set to false and
`resultResp` is populated (`useResultsPane.tsx:130-145`), causing the hook to return
`SingleQueryResultPane` instances (`useResultsPane.tsx:228-258`); until the
`buildV1ChartDataPayload` promise resolves and `setHasPostProcessing(true)` runs,
`hasPostProcessing` remains `false`.

4. In that window, because `!hasPostProcessing && result.rowcount >= effectiveRowLimit` is
true, `onServerSort={handleServerSort}` is passed down (`useResultsPane.tsx:247-253``SingleQueryResultPane.tsx:95-105``GridTable/index.tsx:49-65`), so clicking a column
header sends a server-side `orderby` while `row_limit` is applied to the raw SQL feeding
post-processing, potentially truncating or reordering the input rows for
cumulative/rolling operations and yielding incorrect post-processed values, even though
the design comment in `useResultsPane.tsx:68-72` indicates post-processed queries should
never be server-sorted.

Fix in Cursor | Fix in VSCode Claude

(Use Cmd/Ctrl + Click for best experience)

Prompt for AI Agent 🤖
This is a comment left during a code review.

**Path:** superset-frontend/src/explore/components/DataTablesPane/components/useResultsPane.tsx
**Line:** 73:73
**Comment:**
	*Incorrect Condition Logic: Initializing post-processing detection to `false` temporarily enables server-side sorting before async detection finishes, which can send incorrect `orderby + row_limit` requests for post-processed queries. Start in a disabled/unknown state and enable server sort only after detection confirms no post-processing.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
👍 | 👎

row_limit: effectiveRowLimit,
// A new `orderby` produces a new object, missing the cache below and
// triggering a server-side re-query in the sorted order.
...(orderby.length > 0 && { orderby }),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Omitting orderby when the sort model becomes empty makes a "clear sort" action indistinguishable from "no explicit sort provided". Downstream query builders then re-apply their metric-based default ordering, so the grid can show aria-sort="none" while rows are still server-sorted. Keep an explicit empty orderby (or track a "user sorted" flag) so clearing sort actually clears server ordering. [logic error]

Severity Level: Major ⚠️
- ⚠️ Results pane clear-sort reverts to metric default ordering.
- ⚠️ Grid header shows no sort while backend still orders.
- ⚠️ Users may misinterpret "unsorted" as raw row order.
Steps of Reproduction ✅
1. Open the Explore view and ensure the results data panel is visible so that
`DataTablesPane` renders (entry component at
`superset-frontend/src/explore/components/DataTablesPane/DataTablesPane.tsx:80-90`).

2. With the "Results" tab active, `DataTablesPane` calls `useResultsPane`
(`DataTablesPane.tsx:185-198`), which builds `cappedFormData` without an `orderby` field
when `orderby` state is empty (`useResultsPane.tsx:85-92`).

3. Because no `orderby` is provided in `cappedFormData`, `getChartDataRequest`
(`useResultsPane.tsx:121-128`) sends form data without `orderby`, and the backend query
builders apply their metric-based default ordering as verified by
`tests/unit_tests/mcp_service/chart/test_chart_helpers.py:860-868` (default `orderby` from
first metric when omitted).

4. Click a grid column header to sort: `GridTable` translates the Ag-Grid sort state into
an `orderby` array and calls `onServerSort(orderby)` (`GridTable/index.tsx:57-64`), which
`SingleQueryResultPane` passes through (`SingleQueryResultPane.tsx:97-105`) to
`useResultsPane.handleServerSort` (`useResultsPane.tsx:104-106`), updating `orderby` so
the next request includes an explicit `orderby` (`useResultsPane.tsx:85-92`); clicking the
same header again to clear the sort causes `GridTable` to send an empty `[]`,
`handleServerSort` resets `orderby` to `[]`, `cappedFormData` drops the `orderby` field
entirely (`...(orderby.length > 0 && { orderby })` at `useResultsPane.tsx:91`), and the
subsequent server request again uses the backend's implicit metric-based ordering while
the grid's header shows no active sort.

Fix in Cursor | Fix in VSCode Claude

(Use Cmd/Ctrl + Click for best experience)

Prompt for AI Agent 🤖
This is a comment left during a code review.

**Path:** superset-frontend/src/explore/components/DataTablesPane/components/useResultsPane.tsx
**Line:** 91:91
**Comment:**
	*Logic Error: Omitting `orderby` when the sort model becomes empty makes a "clear sort" action indistinguishable from "no explicit sort provided". Downstream query builders then re-apply their metric-based default ordering, so the grid can show `aria-sort="none"` while rows are still server-sorted. Keep an explicit empty `orderby` (or track a "user sorted" flag) so clearing sort actually clears server ordering.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
👍 | 👎

Comment on lines +104 to +106
const handleServerSort = useCallback((nextOrderby: [string, boolean][]) => {
setOrderby(nextOrderby);
}, []);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: The new server-sort path can fire multiple overlapping requests (e.g., rapid header clicks), but there is no request sequencing or cancellation, so a slower older response can overwrite a newer sort result. Add a request id/AbortController guard and only apply the latest response to prevent stale data races. [race condition]

Severity Level: Critical 🚨
- ❌ Results pane can display rows for an old sort.
- ⚠️ Grid sort indicators may not match underlying row subset.
- ⚠️ Fast header clicks can cause confusing result "flapping."
Steps of Reproduction ✅
1. In Explore, open the results data panel so `DataTablesPane` renders and calls
`useResultsPane` (`DataTablesPane.tsx:80-90` and `185-198`), and ensure the result set is
large enough that `result.rowcount >= effectiveRowLimit` so `onServerSort` is wired
(`useResultsPane.tsx:82-84` and `232-255`).

2. With a non–post-processed chart (so `hasPostProcessing` is false),
`SingleQueryResultPane` passes the `onServerSort` callback from `useResultsPane` down to
`GridTable` (`SingleQueryResultPane.tsx:49-65` and `95-105`), where Ag-Grid's
`onSortChanged` builds an `orderby` array from the current column sort state and invokes
`onServerSort(orderby)` (`GridTable/index.tsx:49-65`).

3. Rapidly click two different column headers (or repeatedly toggle sort) so that
`GridTable` fires multiple `onSortChanged` events in quick succession; for each event,
`handleServerSort` in `useResultsPane` updates `orderby` (`useResultsPane.tsx:104-106`),
which changes `cappedFormData` (`useResultsPane.tsx:85-93`) and triggers the `useEffect`
that starts a new `getChartDataRequest` without cancelling the previous one
(`useResultsPane.tsx:108-147`).

4. If the earlier request's network response arrives after the later one (e.g., the "old"
sort query is slower), its `.then` handler still calls `setResultResp(...)` and
`cache.set(cappedFormData, json.result)` (`useResultsPane.tsx:130-134`), and its
`.finally` sets `isLoading` false (`useResultsPane.tsx:143-145`), overwriting the more
recent sort's data because there is no AbortController or request-id check here (unlike
the guarded, abortable flow in `exploreJSON` at `chartAction.ts:221-249` and the
stale-response dropping at `chartAction.ts:278-79`), leaving the grid showing sort UI for
the last header clicked but row data corresponding to an earlier `orderby`.

Fix in Cursor | Fix in VSCode Claude

(Use Cmd/Ctrl + Click for best experience)

Prompt for AI Agent 🤖
This is a comment left during a code review.

**Path:** superset-frontend/src/explore/components/DataTablesPane/components/useResultsPane.tsx
**Line:** 104:106
**Comment:**
	*Race Condition: The new server-sort path can fire multiple overlapping requests (e.g., rapid header clicks), but there is no request sequencing or cancellation, so a slower older response can overwrite a newer sort result. Add a request id/AbortController guard and only apply the latest response to prevent stale data races.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
👍 | 👎

updateTableOwnState(setDataMask, modifiedOwnState);
},
[serverPagination, serverPaginationData, setDataMask],
[serverPaginationData, setDataMask],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: The callback dependency list is incomplete: it reads serverPagination inside handleSortByChange but does not include it in the useCallback dependencies. If serverPagination flips (for example when result truncation state changes), this callback can keep a stale value and either skip required server re-sorts or keep issuing server sort updates when it should not. Add serverPagination to the dependency array so sorting behavior stays in sync with current mode. [logic error]

Severity Level: Major ⚠️
- ❌ Table chart server-side sorting ignores runtime mode changes.
- ⚠️ Explore users see inconsistent sorting after toggling pagination.
Steps of Reproduction ✅
1. In `superset-frontend/plugins/plugin-chart-table/src/transformProps.ts` lines 521–523,
the table chart's `server_pagination` control from `rawFormData` is mapped into the
boolean `serverPagination` prop returned from `transformProps` and passed into
`TableChart` via `TableChartTransformedProps` (see `src/types.ts` lines 5–13 where
`serverPagination: boolean; serverPaginationData; setDataMask` are defined).

2. In `superset-frontend/plugins/plugin-chart-table/src/TableChart.tsx` lines 392–52,
`TableChart` destructures `serverPagination = false`, `serverPaginationData`, and
`setDataMask` from `props`, and later defines `handleSortByChange` at lines 54–66 (from
the `Read` output) as:

   - `if (!serverPagination) return;`

   - build `modifiedOwnState` from `serverPaginationData` with the new `sortBy` and
   `currentPage: 0`

   - call `updateTableOwnState(setDataMask, modifiedOwnState)`

   This callback is memoized with `useCallback` and a dependency array that currently only
   includes `[serverPaginationData, setDataMask]` (line 1545 in the diff).

3. When a user toggles the "Server pagination" control for a table chart in Explore,
`transformProps` recomputes props with `serverPagination` flipped from `false` to `true`
(or vice versa) while keeping the same `serverPaginationData` object and `setDataMask`
hook reference (see `transformProps.ts` lines 36–43 and 39–41 where `serverPagination`
controls whether `serverPaginationData` comes from ownState or defaults). Because
`useCallback` in `TableChart.tsx` depends only on `[serverPaginationData, setDataMask]`,
React reuses the previous `handleSortByChange` function instance, whose closure still
captures the old `serverPagination` boolean.

4. After this toggle, when the user clicks a column header to sort, the `DataTable`
component in `superset-frontend/plugins/plugin-chart-table/src/DataTable/DataTable.tsx`
lines 142–162 detects a sort change (`sortBy`) and, if `serverPagination` is true, calls
`handleSortByChange`:

   - `const serverSortBy = serverPaginationData?.sortBy || [];`

   - `if (serverPagination && !isEqual(sortBy, serverSortBy)) { ...
   handleSortByChange([...]); }`

   At this point `DataTable` sees `serverPagination === true` (new prop) and invokes
   `handleSortByChange`, but inside that callback the captured `serverPagination` is still
   `false`, so it immediately returns and never calls `updateTableOwnState`. As a result,
   no server-side `sortBy` is pushed into `serverPaginationData`, the chart-data request
   is not reissued, and the "server-side column sorting" path described in this PR
   silently fails after toggling server pagination at runtime. Adding `serverPagination`
   to the `useCallback` dependency array forces `handleSortByChange` to be recreated when
   the mode changes, keeping the sort behavior in sync with the current pagination mode.

Fix in Cursor | Fix in VSCode Claude

(Use Cmd/Ctrl + Click for best experience)

Prompt for AI Agent 🤖
This is a comment left during a code review.

**Path:** superset-frontend/plugins/plugin-chart-table/src/TableChart.tsx
**Line:** 1545:1545
**Comment:**
	*Logic Error: The callback dependency list is incomplete: it reads `serverPagination` inside `handleSortByChange` but does not include it in the `useCallback` dependencies. If `serverPagination` flips (for example when result truncation state changes), this callback can keep a stale value and either skip required server re-sorts or keep issuing server sort updates when it should not. Add `serverPagination` to the dependency array so sorting behavior stays in sync with current mode.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
👍 | 👎

Comment on lines +397 to +401
if isinstance(json_body, dict) and json_body.get("queries")
else []
)
sort_error = validate_sort_params(orderby)
if sort_error is not None:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: The new sort-direction validation only inspects queries[0], so any additional query objects in the same request bypass this strict boolean check. This creates inconsistent behavior where malformed sort directions in later queries are silently accepted (or coerced) instead of being rejected. Validate orderby for every entry in json_body["queries"], not just the first one. [incomplete implementation]

Severity Level: Major ⚠️
- ⚠️ Multi-query chart requests have inconsistent sort validation.
- ⚠️ Later queries can carry non-boolean sort directions.
- ⚠️ Downstream SQL builders see unexpected `orderby` value types.
- ⚠️ API contract "boolean sort direction" only partially enforced.
Steps of Reproduction ✅
1. Note that the Chart Data endpoint `ChartDataRestApi.data` in
`superset/charts/data/api.py:93-200` accepts a query context JSON body where `"queries"`
is documented as "one or many query objects" (docstring lines 115–116).

2. Observe that this method computes `orderby` using only the first query: at
`superset/charts/data/api.py:152-161`, it assigns `orderby = (json_body.get("queries",
[{}])[0].get("orderby", []) if isinstance(json_body, dict) and json_body.get("queries")
else [])` and then calls `sort_error = validate_sort_params(orderby)`.

3. Note that the query context is built from *all* query objects:
`QueryContextFactory.create` in `superset/common/query_context_factory.py:45-89` iterates
over every element in the incoming `queries` list and passes each `query_obj` dict
(including its `orderby`) into `QueryObjectFactory.create`, producing a list of
`QueryObject` instances assigned to `QueryContext.queries`.

4. Craft a POST request to the `api/v1/chart/data` endpoint (which is wired to
`ChartDataRestApi.data` via `ChartRestApi` in `superset/charts/api.py`) with a JSON body
like:

   `{"datasource": {...}, "queries": [{"orderby": [("metric1", true)]}, {"orderby":
   [("metric2", "asc")]}, ...], "result_type": "full", "form_data": {...}}`. The first
   query's `orderby` uses a valid boolean direction, while the second query uses the
   invalid string `"asc"`.

5. When this request reaches `ChartDataRestApi.data`, the first query's `orderby` list is
extracted and passed to `validate_sort_params`, which returns `None` because all
directions in that list are booleans (see
`superset/tests/unit_tests/charts/data/test_api.py:13-20` where valid boolean directions
are accepted). No validation is ever performed on the second query's `orderby` list, so
the non-boolean `"asc"` is never rejected.

6. The method then constructs a `ChartDataCommand` with a `QueryContext` that includes
*all* queries, including the second query whose `orderby` contains a non-boolean
direction. Since `QueryObject` in `superset/common/query_object.py:112-152` simply assigns
`self.orderby = orderby or []` without type-checking, this malformed sort directive is
accepted and propagated, meaning later queries bypass the sort-direction validation that
the new helper `validate_sort_params` was meant to enforce.

7. This leads to inconsistent behavior: malformed sort directions in the first query
result in an immediate HTTP 400 from `validate_sort_params`, while the same malformed
directions in any subsequent query are silently accepted and only expressed (as
mis-sorting or backend errors) deeper in the query pipeline instead of being rejected
uniformly at the API boundary.

Fix in Cursor | Fix in VSCode Claude

(Use Cmd/Ctrl + Click for best experience)

Prompt for AI Agent 🤖
This is a comment left during a code review.

**Path:** superset/charts/data/api.py
**Line:** 397:401
**Comment:**
	*Incomplete Implementation: The new sort-direction validation only inspects `queries[0]`, so any additional query objects in the same request bypass this strict boolean check. This creates inconsistent behavior where malformed sort directions in later queries are silently accepted (or coerced) instead of being rejected. Validate `orderby` for every entry in `json_body["queries"]`, not just the first one.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
👍 | 👎

@bito-code-review bito-code-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Agent Run #0a5875

Actionable Suggestions - 4
  • superset-frontend/plugins/plugin-chart-table/src/buildQuery.ts - 1
  • superset-frontend/plugins/plugin-chart-table/src/TableChart.tsx - 1
  • superset-frontend/src/components/GridTable/types.ts - 1
    • Doc/impl mismatch on multi-column sort · Line 63-68
  • superset-frontend/plugins/plugin-chart-ag-grid-table/src/buildQuery.ts - 1
Additional Suggestions - 4
  • superset/charts/data/api.py - 1
    • Inconsistent validation across endpoints · Line 390-404
      The `data()` method now validates `orderby` direction BEFORE schema loading via `_create_query_context_from_form()`. However, `get_data()` (line ~292) and `data_from_cache()` (line ~488) still rely solely on marshmallow schema validation in `_create_query_context_from_form()`. This creates two inconsistent code paths for the same input validation, producing different error responses (`SORT_DIRECTION_INVALID` vs generic schema `ValidationError`).
  • tests/unit_tests/models/helpers_test.py - 2
    • Semantic duplication in test helpers · Line 2709-2740
      The `_orderby_sql` helper at lines 2709-2740 duplicates the SqlaTable/TableColumn setup pattern that appears directly in two other tests in the same diff. This creates divergence risk if one copy is updated without the other. Consider whether the helper's abstraction adds value or if direct setup better serves maintainability.
    • Fragile assertion using string index · Line 2837-2839
      The assertions at lines 2837-2839 use `sql.index()` to verify WHERE appears before ORDER BY. This approach is fragile if SQL formatting changes (whitespace, newlines) or if dialect-specific SQL adds clauses between WHERE and ORDER BY.
  • superset-frontend/src/components/GridTable/types.ts - 1
    • Docs mismatch on multi-column sort · Line 63-68
      The JSDoc for `onServerSort` (lines 63-68) states 'Only the primary sort column is included, matching the chart data API's single-column sort constraint,' but the implementation (lines 57-64 in index.tsx) correctly handles multi-column sorting using `sortIndex`. This documentation mismatch could mislead consumers into believing multi-column sorts are not supported when they are.
Review Details
  • Files reviewed - 20 · Commit Range: d13ed02..9b101ed
    • superset-frontend/plugins/plugin-chart-ag-grid-table/src/buildQuery.ts
    • superset-frontend/plugins/plugin-chart-table/src/TableChart.tsx
    • superset-frontend/plugins/plugin-chart-table/src/buildQuery.ts
    • superset-frontend/plugins/plugin-chart-table/test/TableChart.test.tsx
    • superset-frontend/src/components/Chart/DrillDetail/DrillDetailPane.tsx
    • superset-frontend/src/components/GridTable/index.tsx
    • superset-frontend/src/components/GridTable/types.ts
    • superset-frontend/src/explore/components/DataTablesPane/components/SingleQueryResultPane.tsx
    • superset-frontend/src/explore/components/DataTablesPane/components/useResultsPane.tsx
    • superset-frontend/src/explore/components/DataTablesPane/types.ts
    • superset/charts/data/api.py
    • superset/common/query_actions.py
    • superset/views/datasource/schemas.py
    • superset/views/datasource/utils.py
    • tests/integration_tests/datasource_tests.py
    • tests/unit_tests/charts/data/__init__.py
    • tests/unit_tests/charts/data/test_api.py
    • tests/unit_tests/common/test_query_actions.py
    • tests/unit_tests/models/helpers_test.py
    • tests/unit_tests/views/datasource/utils_test.py
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • Eslint (Linter) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

// Fall back to a metric-based default sort only when no explicit orderby
// was supplied (e.g. a column sort from the "View as table" results pane).
// An explicit orderby from form data takes precedence.
if (orderby.length === 0) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing test coverage for orderby logic

The new if (orderby.length === 0) guard changes behavior to preserve explicit orderby, but buildQuery.test.ts has no tests validating this logic. Add tests covering: (a) explicit orderby preserved when sortByMetric exists, (b) explicit orderby preserved when metrics exist, (c) defaults applied only when orderby is empty.

Code Review Run #0a5875


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

updateTableOwnState(setDataMask, modifiedOwnState);
},
[serverPagination, serverPaginationData, setDataMask],
[serverPaginationData, setDataMask],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing dependency in useCallback

The handleSortByChange callback captures serverPagination in the function body (line 1535: if (!serverPagination) return), but this variable is missing from the dependency array. This creates a stale closure that can cause the callback to use an outdated serverPagination value after prop changes.

Code suggestion
Check the AI-generated fix before applying
 --- a/superset-frontend/plugins/plugin-chart-table/src/TableChart.tsx
 +++ b/superset-frontend/plugins/plugin-chart-table/src/TableChart.tsx
 @@ -1542,7 +1542,7 @@ export default function TableChart<D extends DataRecord = DataRecord>(
        };
        updateTableOwnState(setDataMask, modifiedOwnState);
      },
 -    [serverPaginationData, setDataMask],
 +    [serverPagination, serverPaginationData, setDataMask],
    );
 
      const handleSearch = (searchText: string) => {

Code Review Run #0a5875


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

Comment on lines +63 to +68
/**
* Called when the sort state changes, with it translated to a query
* `orderby` (`[[columnId, isAscending]]`) so the consumer can re-request
* server-sorted data. Only the primary sort column is included, matching the
* chart data API's single-column sort constraint.
*/

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doc/impl mismatch on multi-column sort

Update the JSDoc for onServerSort to clarify that when multi-column sorting is used, all sorted columns are passed to the callback in priority order, matching the implementation.

Code Review Run #0a5875


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

// Fall back to a metric-based default sort only when no explicit orderby
// was supplied (e.g. a column sort from the "View as table" results pane).
// An explicit orderby from form data takes precedence.
if (orderby.length === 0) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing test for explicit orderby

The new guard if (orderby.length === 0) correctly prevents overriding explicit orderby from form data. However, no test covers this scenario — existing tests use basicFormData without explicit orderby. Add a test with orderby: [['state', true]] alongside sortByMetric to verify the explicit orderby is preserved.

Code Review Run #0a5875


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds server-side column sorting to Superset’s paginated/truncated table experiences so header-click sorting reflects the full dataset (not just the currently loaded page), and wires the same behavior through Drill to detail via /datasource/samples.

Changes:

  • Frontend: propagate header sort state as orderby and re-fetch data from the server for “View as table” results and drill-detail tables (with safeguards to fall back to client-side sort for post-processed queries).
  • Backend: accept/preserve orderby for drill-detail, omit orderby from the COUNT(*) query, and add sort-direction validation on the chart-data endpoint.
  • Tests: add unit/integration coverage for orderby behavior in chart data validation, query actions, and datasource samples.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tests/unit_tests/views/datasource/utils_test.py Adds unit test asserting orderby is forwarded only to the data query (not COUNT(*)) for samples.
tests/unit_tests/models/helpers_test.py Adds unit tests verifying orderby compiles into SQL safely and errors on unknown columns.
tests/unit_tests/common/test_query_actions.py Adds unit tests ensuring drill-detail preserves caller-supplied orderby and defaults when absent.
tests/unit_tests/charts/data/test_api.py Adds unit tests for new chart-data sort-direction validator.
tests/unit_tests/charts/data/init.py Creates package init for new chart-data unit tests.
tests/integration_tests/datasource_tests.py Adds integration test validating /datasource/samples honors orderby and produces distinct cache keys.
superset/views/datasource/utils.py Omits orderby from the COUNT(*) query to avoid meaningless/invalid ordering for counts.
superset/views/datasource/schemas.py Extends samples payload schema to accept orderby.
superset/common/query_actions.py Preserves supplied orderby in drill-detail; only defaults when missing.
superset/charts/data/api.py Adds validate_sort_params and invokes it from POST /api/v1/chart/data.
superset-frontend/src/explore/components/DataTablesPane/types.ts Adds onServerSort + isLoading props to support in-place refetch while preserving grid state.
superset-frontend/src/explore/components/DataTablesPane/components/useResultsPane.tsx Tracks orderby, disables server sort for post-processing, and keeps pane mounted during refetch.
superset-frontend/src/explore/components/DataTablesPane/components/SingleQueryResultPane.tsx Threads onServerSort + isLoading down to GridTable.
superset-frontend/src/components/GridTable/types.ts Adds onServerSort callback to translate grid sort state into orderby.
superset-frontend/src/components/GridTable/index.tsx Emits orderby (including multi-column priority ordering) on sort changes.
superset-frontend/src/components/Chart/DrillDetail/DrillDetailPane.tsx Implements server-side sorting for drill-detail table headers and resets paging/cache on sort changes.
superset-frontend/plugins/plugin-chart-table/test/TableChart.test.tsx Adds accessibility regression test for sortable headers exposing aria-sort.
superset-frontend/plugins/plugin-chart-table/src/TableChart.tsx Adds aria-sort to headers; resets to first page when sort changes.
superset-frontend/plugins/plugin-chart-table/src/buildQuery.ts Applies metric-default sorting only when no explicit orderby is provided.
superset-frontend/plugins/plugin-chart-ag-grid-table/src/buildQuery.ts Mirrors the same “respect explicit orderby” behavior for ag-grid table plugin.

Comment on lines +63 to +68
/**
* Called when the sort state changes, with it translated to a query
* `orderby` (`[[columnId, isAscending]]`) so the consumer can re-request
* server-sorted data. Only the primary sort column is included, matching the
* chart data API's single-column sort constraint.
*/
Comment on lines 1533 to 1546
const handleSortByChange = useCallback(
(sortBy: SortByItem[]) => {
if (!serverPagination) return;
const modifiedOwnState = {
...serverPaginationData,
sortBy,
// Changing the sort re-queries the full dataset, so the
// previous page offset is meaningless — return to the first page.
currentPage: 0,
};
updateTableOwnState(setDataMask, modifiedOwnState);
},
[serverPagination, serverPaginationData, setDataMask],
[serverPaginationData, setDataMask],
);
Comment on lines 391 to +402
try:
query_context = self._create_query_context_from_form(json_body)
# Validate sort parameters before executing the query so bad sort
# directions are rejected early, before the query builder runs.
orderby = (
json_body.get("queries", [{}])[0].get("orderby", [])
if isinstance(json_body, dict) and json_body.get("queries")
else []
)
sort_error = validate_sort_params(orderby)
if sort_error is not None:
return sort_error
Comment on lines +95 to +100
metadata={
"description": "Expects a list of lists where the first element is the "
"column name to sort by, and the second element is a boolean "
"(true = ascending).",
"example": [("my_col_1", False), ("my_col_2", True)],
},
@codecov

codecov Bot commented Jun 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 63.41463% with 30 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.13%. Comparing base (443fd7b) to head (9b101ed).
⚠️ Report is 27 commits behind head on master.

Files with missing lines Patch % Lines
...c/components/Chart/DrillDetail/DrillDetailPane.tsx 38.88% 11 Missing ⚠️
...perset-frontend/src/components/GridTable/index.tsx 10.00% 9 Missing ⚠️
superset/charts/data/api.py 66.66% 2 Missing and 2 partials ⚠️
...tend/plugins/plugin-chart-table/src/TableChart.tsx 66.66% 2 Missing ⚠️
...nents/DataTablesPane/components/useResultsPane.tsx 90.90% 2 Missing ⚠️
...ugins/plugin-chart-ag-grid-table/src/buildQuery.ts 80.00% 1 Missing ⚠️
...ntend/plugins/plugin-chart-table/src/buildQuery.ts 80.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #40907      +/-   ##
==========================================
- Coverage   64.13%   64.13%   -0.01%     
==========================================
  Files        2653     2653              
  Lines      143601   143668      +67     
  Branches    33132    33161      +29     
==========================================
+ Hits        92104    92139      +35     
- Misses      49884    49912      +28     
- Partials     1613     1617       +4     
Flag Coverage Δ
hive 39.50% <20.00%> (-0.01%) ⬇️
javascript 67.87% <61.19%> (-0.01%) ⬇️
mysql 58.24% <73.33%> (+<0.01%) ⬆️
postgres 58.31% <73.33%> (-0.01%) ⬇️
presto 41.10% <20.00%> (-0.01%) ⬇️
python 59.79% <73.33%> (-0.01%) ⬇️
sqlite 57.93% <73.33%> (+<0.01%) ⬆️
unit 100.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api Related to the REST API plugins size/XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants